{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 南瓜定价的线性回归和多项式回归预测 - 第3课\n",
    "\n",
    "加载所需的库和数据集。将数据转换为包含以下子集的数据框：\n",
    "\n",
    "- 仅获取按蒲式耳定价的南瓜  \n",
    "- 将日期转换为月份  \n",
    "- 计算高价和低价的平均价格  \n",
    "- 将价格转换为按蒲式耳数量反映的定价  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from datetime import datetime\n",
    "\n",
    "pumpkins = pd.read_csv('../data/US-pumpkins.csv')\n",
    "pumpkins.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "pumpkins = pumpkins[pumpkins['Package'].str.contains('bushel', case=True, regex=True)]\n",
    "\n",
    "new_columns = ['Package', 'Variety', 'City Name', 'Month', 'Low Price', 'High Price', 'Date']\n",
    "pumpkins = pumpkins.drop([c for c in pumpkins.columns if c not in new_columns], axis=1)\n",
    "\n",
    "price = (pumpkins['Low Price'] + pumpkins['High Price']) / 2\n",
    "\n",
    "month = pd.DatetimeIndex(pumpkins['Date']).month\n",
    "day_of_year = pd.to_datetime(pumpkins['Date']).apply(lambda dt: (dt-datetime(dt.year,1,1)).days)\n",
    "\n",
    "new_pumpkins = pd.DataFrame(\n",
    "    {'Month': month, \n",
    "     'DayOfYear' : day_of_year, \n",
    "     'Variety': pumpkins['Variety'], \n",
    "     'City': pumpkins['City Name'], \n",
    "     'Package': pumpkins['Package'], \n",
    "     'Low Price': pumpkins['Low Price'],\n",
    "     'High Price': pumpkins['High Price'], \n",
    "     'Price': price})\n",
    "\n",
    "new_pumpkins.loc[new_pumpkins['Package'].str.contains('1 1/9'), 'Price'] = price/1.1\n",
    "new_pumpkins.loc[new_pumpkins['Package'].str.contains('1/2'), 'Price'] = price*2\n",
    "\n",
    "new_pumpkins.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "散点图提醒我们，数据集中仅包含从8月到12月的月份数据。我们可能需要更多数据才能以线性方式得出结论。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "new_pumpkins.plot.scatter('Month','Price')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "new_pumpkins.plot.scatter('DayOfYear','Price')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "让我们看看是否存在相关性："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "print(new_pumpkins['Month'].corr(new_pumpkins['Price']))\n",
    "print(new_pumpkins['DayOfYear'].corr(new_pumpkins['Price']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`corr` 方法是 Pandas 提供的用于计算两个列之间 **相关系数** 的方法。相关系数（Correlation Coefficient）是一个衡量两个变量之间线性关系的统计指标，取值范围为 -1 到 1：\n",
    "\n",
    "- **1** 表示完全正相关（当一个变量增加时，另一个变量也以相同比例增加）。\n",
    "- **-1** 表示完全负相关（当一个变量增加时，另一个变量以相同比例减少）。\n",
    "- **0** 表示没有线性相关关系。\n",
    "\n",
    "在你的代码中：\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "print(new_pumpkins['Month'].corr(new_pumpkins['Price']))\n",
    "print(new_pumpkins['DayOfYear'].corr(new_pumpkins['Price']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看起来相关性很小，但图中的价格点似乎有几个明显的聚类，这表明可能存在其他更重要的关系。让我们绘制一张图，显示不同的南瓜品种： "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "ax=None\n",
    "colors = ['red','blue','green','yellow']\n",
    "for i,var in enumerate(new_pumpkins['Variety'].unique()):\n",
    "    ax = new_pumpkins[new_pumpkins['Variety']==var].plot.scatter('DayOfYear','Price',ax=ax,c=colors[i],label=var)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 173,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "new_pumpkins.groupby('Variety')['Price'].mean().plot(kind='bar')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the time being, let's concentrate only on one variety - **pie type**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 174,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "pie_pumpkins = new_pumpkins[new_pumpkins['Variety']=='PIE TYPE']\n",
    "print(pie_pumpkins['DayOfYear'].corr(pie_pumpkins['Price']))\n",
    "pie_pumpkins.plot.scatter('DayOfYear','Price')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 线性回归\n",
    "\n",
    "我们将使用 Scikit Learn 来训练线性回归模型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 175,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.metrics import r2_score, mean_squared_error, mean_absolute_error\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "X = pie_pumpkins['DayOfYear'].to_numpy().reshape(-1,1)\n",
    "y = pie_pumpkins['Price']\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)\n",
    "lin_reg = LinearRegression()\n",
    "lin_reg.fit(X_train,y_train)\n",
    "\n",
    "pred = lin_reg.predict(X_test)\n",
    "\n",
    "mse = np.sqrt(mean_squared_error(y_test,pred))\n",
    "print(f'Mean error: {mse:3.3} ({mse/np.mean(pred)*100:3.3}%)')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "plt.scatter(X_test,y_test)\n",
    "plt.plot(X_test,pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The slope of the line can be determined from linear regression coefficients:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 178,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "lin_reg.coef_, lin_reg.intercept_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们可以使用训练好的模型来预测价格："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 179,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "# Pumpkin price on programmer's day\n",
    "\n",
    "lin_reg.predict([[256]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 多项式回归\n",
    "\n",
    "Sometimes the relationship between features and the results is inherently non-linear. For example, pumpkin prices might be high in winter (months=1,2), then drop over summer (months=5-7), and then rise again. Linear regression is unable to fin this relationship accurately.\n",
    "\n",
    "In this case, we may consider adding extra features. Simple way is to use polynomials from input features, which would result in **polynomial regression**. In Scikit Learn, we can automatically pre-compute polynomial features using pipelines: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "from sklearn.pipeline import make_pipeline\n",
    "\n",
    "pipeline = make_pipeline(PolynomialFeatures(2), LinearRegression())\n",
    "\n",
    "pipeline.fit(X_train,y_train)\n",
    "\n",
    "pred = pipeline.predict(X_test)\n",
    "\n",
    "mse = np.sqrt(mean_squared_error(y_test,pred))\n",
    "print(f'Mean error: {mse:3.3} ({mse/np.mean(pred)*100:3.3}%)')\n",
    "\n",
    "score = pipeline.score(X_train,y_train)\n",
    "print('Model determination: ', score)\n",
    "\n",
    "plt.scatter(X_test,y_test)\n",
    "plt.plot(sorted(X_test),pipeline.predict(sorted(X_test)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 编码品种\n",
    "\n",
    "在理想情况下，我们希望能够使用相同的模型预测不同南瓜品种的价格。为了考虑品种，我们首先需要将其转换为数值形式，或 **编码**。有几种方法可以实现：\n",
    "\n",
    "* 简单的数值编码，这将构建一个不同品种的表，然后用该表中的索引替换品种名称。这对于线性回归来说不是最佳选择，因为线性回归会将索引的数值值纳入考虑，而数值值可能与价格没有数值相关性。\n",
    "* 独热编码（One-hot encoding），这将用 4 个不同的列替换 `Variety` 列，每列对应一个品种，如果对应行是给定品种，则该列值为 1，否则为 0。\n",
    "\n",
    "以下代码展示了如何对品种进行独热编码："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 181,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "pd.get_dummies(new_pumpkins['Variety'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 品种的线性回归 \n",
    "\n",
    "我们现在将使用与上述相同的代码，但输入不是 `DayOfYear`，而是我们独热编码后的品种："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 182,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = pd.get_dummies(new_pumpkins['Variety'])\n",
    "y = new_pumpkins['Price']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 183,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "def run_linear_regression(X,y):\n",
    "    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)\n",
    "    lin_reg = LinearRegression()\n",
    "    lin_reg.fit(X_train,y_train)\n",
    "\n",
    "    pred = lin_reg.predict(X_test)\n",
    "\n",
    "    mse = np.sqrt(mean_squared_error(y_test,pred))\n",
    "    print(f'Mean error: {mse:3.3} ({mse/np.mean(pred)*100:3.3}%)')\n",
    "\n",
    "    score = lin_reg.score(X_train,y_train)\n",
    "    print('Model determination: ', score)\n",
    "\n",
    "run_linear_regression(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们还可以尝试以相同的方式使用其他特征，并将它们与数值特征（如 `Month` 或 `DayOfYear`）结合起来："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 184,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "X = pd.get_dummies(new_pumpkins['Variety']) \\\n",
    "        .join(new_pumpkins['Month']) \\\n",
    "        .join(pd.get_dummies(new_pumpkins['City'])) \\\n",
    "        .join(pd.get_dummies(new_pumpkins['Package']))\n",
    "y = new_pumpkins['Price']\n",
    "\n",
    "run_linear_regression(X,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 多项式回归\n",
    "\n",
    "多项式回归也可以与独热编码的分类特征一起使用。训练多项式回归的代码与我们上面看到的基本相同。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 185,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": []
    }
   ],
   "source": [
    "from sklearn.preprocessing import PolynomialFeatures\n",
    "from sklearn.pipeline import make_pipeline\n",
    "\n",
    "pipeline = make_pipeline(PolynomialFeatures(2), LinearRegression())\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)\n",
    "\n",
    "pipeline.fit(X_train,y_train)\n",
    "\n",
    "pred = pipeline.predict(X_test)\n",
    "\n",
    "mse = np.sqrt(mean_squared_error(y_test,pred))\n",
    "print(f'Mean error: {mse:3.3} ({mse/np.mean(pred)*100:3.3}%)')\n",
    "\n",
    "score = pipeline.score(X_train,y_train)\n",
    "print('Model determination: ', score)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  },
  "metadata": {
   "interpreter": {
    "hash": "70b38d7a306a849643e446cd70466270a13445e5987dfa1344ef2b127438fa4d"
   }
  },
  "orig_nbformat": 2
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
